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Background / Context: 

The low level of mathematics performance of U.S. students in relation to national standards and 
in international comparisons has concerned educators and policy makers for many years 
(National Research Council, 2001). Though a slight improvement from previous years, results 
from the latest NAEP (National Center for Education Statistics, 2011) indicate that only 40% of 
fourth-grade students were deemed at or above proficient and that 18% were below basic. Signs 
of these problems appear early and are long-lasting (Aud et ah, 2010; Morgan, Farkas, & Wu, 
2009). Even at school entry, significant differences in student knowledge reliably appear on 
measures related to counting principles and number knowledge as well as more complex 
understandings of quantities, operations, and problem solving (Griffin, Case, & Siegler, 1994; 
Jordan, Kaplan, & Locuniak, 2006). 

Though it is desirable for students to have the foundational mathematics knowledge they need to 
succeed when exposed to mathematics instruction in the early elementary grades, the reality is 
that many do not (Bodovski & Farkas, 2007; Duncan et al., 2007; Hanich, Jordan, Kaplan, & 
Dick, 2001; Morgan, Farkas, & Wu, 2009). Many struggling students’ difficulties resonate with 
specific aspects of whole numbers, particularly in developing number sense, acquiring efficient 
counting strategies, and retrieving number combinations (Geary, 1993; Gersten, Jordan, & Flojo, 
2005). 

In line with Response to Intervention service delivery models, our research group aims to 
alleviate and prevent mathematics learning difficulties through effective, research-based 
instruction and intervention (Baker, Fien, & Baker, 2010). We posit that inclusion of research- 
based design elements enhances such mathematics instruction by building students’ deep 
understanding of essential early whole number concepts and principles and procedural fluency. 

With funding from an IES Development and Innovation grant (Baker, Clarke, Fien, & Chard, 
2009), our group has developed FUSION, a Tier 2 mathematics intervention program for Grade 1 
students at risk for mathematics difficulties. Our primary goals were to design a feasible and 
usable intervention and gather data on the promise of the intervention to foster students’ 
conceptual understanding of whole number concepts and skills and procedural fluency. The 
FUSION curriculum was developed using an iterative design process aligned with design 
experiment methodology (Brown, 1992; Shavelson, Phillips, Towne, & Feuer, 2003). Brief 
Learning Trials in Year 1, as well as a feasibility study in Year 2 identified needed revisions in 
content, lesson length, structure, and formatting. 

Focus of Study: 

The culminating Year 3 pilot study reported here was intended to measure the extent to which 
the FUSION Tier 2 math intervention shows promise of improving math achievement of first- 
grade students at-risk for math difficulties. 

Setting: 

The pilot study took place in nine schools in two suburban school districts in the northwest. One 
district serves 10,796 students: 33% are minorities, 6% are English learners, 60% are eligible for 
free/reduced lunch, and 15% receive special education services. The other district serves 5,866 
students: 28% are minorities, 3% are English learners, 55% are eligible for free/reduced lunch, 
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and 17% receive special education services. Several schools in each district receive Title- 1 
funds. 

Population / Participants / Subjects: 

A total of 151 students (68 female) participated. Nine district-employed interventionists (all 
female) provided the FUSION instruction. District-provided data revealed 21% of the students in 
our sample were minorities, 16% were English learners, 22% received special education 
services, and 65% were eligible for free/reduced lunch. 

These students were selected through a multi-step screening process of all first graders. 

Screening occurred three months into the school year. Within each participating school, brief 
measures of number sense (EN-CBM, described below) were administered to identify the 
students with the lowest mathematics perfonnance. Need was further verified by teacher 
recommendation and rating of performance in classroom mathematics. FUSION-eligible students 
were required to speak English, regularly attended school, could identify numbers 1-10 and 
could count from 1-10. 

At each school, the at-risk students were randomly assigned to one of two conditions: FUSION 
Intervention or At-risk Control. Additional students were randomly selected to participate in an 
Average Control condition. There were generally fifteen students participating at each school 
(i.e., our goal was five students in each of the three conditions), however thirty students (ten in 
each of the three conditions) participated at one school that operated two FUSION groups. 

Intervention: 

The FUSION program is a Grade 1 (Tier 2) mathematics intervention that focuses specifically on 
building students’ early knowledge of whole number concepts. Four math strands comprise the 
program: (a) base- 10/place value, (b) basic number combinations, (c) multi-digit addition and 
subtraction without renaming, and (d) word problems. Each strand reflects the critical content of 
first grade mathematics (National Governors Association Center for Best Practices, 2010; 
National Council of Teachers of Mathematics, 2006) and aligns with the recommendations of the 
National Math Advisory Panel (2008) and other experts in the field (Kilpatrick, Swafford, & 
Findell 2001; Wu, 2009). FUSION" s 60 scripted lessons utilize an explicit instructional format. 
Lessons contain teacher modeling, scaffolded instructional examples, and opportunities for 
academic feedback. Lessons incorporate a variety of math models and offer frequent 
opportunities for student practice and judicious review. 

FUSION instruction occurred outside of students’ core math and reading time. Lessons lasted 
approximately 30 minutes and were delivered in small-group instructional fonnats, with 
approximately 4-5 students per group. Interventionists were encouraged to complete one lesson 
per day, three times per week. 

Research Design: 

The current study utilized a randomized control trial to examine the promise of the FUSION 
intervention. Within school, students detennined at-risk for math difficulties were randomly 
assigned to one of two conditions, FUSION Intervention or At-risk Control. Additional students 
were randomly selected to participate in an Average Control condition. FUSION students 
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received standard district whole class instruction as well as the FUSION intervention (taught by 
school personnel). Both groups of control students received standard district whole class 
instruction. They were not precluded from receiving other services. Student achievement is our 
primary outcome. 

Data Collection and Analysis: 

Student outcomes. We included both proximal and distal outcome measures targeting students’ 
conceptual understanding and procedural fluency with whole numbers. During the screening 
process and again at posttest, all participants completed a group-administered modification of the 
Early Numeracy - Curriculum-Based Measures (EN-CBM; Clarke & Shinn, 2004) assessing 
quantity discrimination and identifying missing numbers in a sequence. At pretest and posttest, 
all participants completed the unmodified, individually-administered EN-CBM assessing number 
identification, quantity discrimination, identifying missing numbers, and oral counting as well as 
the group-administered Stanford Achievement Test - Tenth Edition (SAT-10) problem solving 
and procedures subtests. Students in the two at-risk conditions also completed ProFUSION, a 
researcher-developed assessment addressing key content in the FUSION intervention. Students 
were asked to write numbers from dictation, identify numbers and numbers missing from a 
sequence, write numbers matching base ten block models, decompose double digit numbers, and 
complete story problems as well as timed and untimed addition/sub traction problems. 

FUSION instruction observation data. We conducted three formal observations of FUSION 
instruction for each group. Observations were coded using the FUSION observation instrument 
and the Ratings of Classroom Management and Instructional Support (RCMIS; Doabler & 
Nelson- Walker, 2009). The FUSION observation instrument measures implementation fidelity of 
the FUSION program. For each activity (4-5 activities per lesson), observers documented the 
absence or presence of instructional design principles specified by the FUSION development 
team (e.g., models the skill or concept, provides group response opportunities, provides 
academic feedback). From this measure, we calculated the percentage of design principles 
implemented by FUSION interventionists. An overall rating of fidelity of implementation was 
recorded using a 7-point scale (0 = low, 7 = high). The RCMIS measures the quality of 
classroom instruction (11 items) across three domains: learning environment, classroom 
management, and the delivery of instruction. At the conclusion of each observation, observers 
recorded their overall impressions of 1 1 features of instructional quality using a 4-point holistic 
rating scale (1 = not present, 2 = somewhat present, 3 = present, and 4 = highly present). Inter- 
rater reliability was estimated as intraclass correlation coefficients ranging from .46 to .60. 
Current studies are beginning to empirically link the three RCMIS domains with increased 
student math achievement (Doabler, Baker, Kosty, Smolkowski, Clarke, Miller & Fien, 2012). 

Fogistics and perceptions surveys. FUSION interventionists and first-grade classroom teachers 
completed online surveys on specific features of the FUSION program, FUSION ’ s effect on 
students’ mathematics engagement and knowledge, the screening process, and the “fit” of the 
curriculum in the school context. 

Results: 

Previous findings from the project’s implementation studies suggest that FUSION is a feasible 
and usable Tier 2 intervention that can be implemented in authentic educational settings. As well, 
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data generated through a single case design (Doabler, Strand Cary, Clarke, Fien, Baker, & 
Jungjohann, 2012) also indicate the promise of FUSION to improve student math achievement. 

Data for the Pilot Study described here have been collected and are currently being analyzed. We 
are particularly interested in whether at-risk students receiving FUSION outperform at-risk 
students who did not receive FUSION and whether that perfonnance reduces the gap between at- 
risk students and the “average” peers. We are also examining whether interventionists reported 
FUSION to be feasible and usable and whether gains are related to the fidelity with which 
interventionists implemented FUSION. We expect to find FUSION participation is associated 
with increased student perfonnance on proximal measures (i.e., ProFUSION and EN-CBMs) and 
distal measures of student achievement (i.e., SAT-10). 

Promising Pilot Study results coupled with previous findings will indicate we achieved our aim 
of creating a first-grade. Tier 2 mathematics intervention that is feasible and usable in schools 
and shows promise for increasing student achievement. 

Conclusions: 

This pilot study was designed to test the extent to which the FUSION intervention improves 
student achievement and is appropriate for authentic first-grade contexts. Understanding student 
outcomes, program implementation fidelity, and interventionists’ perceptions of- and reactions to 
- the intervention has been a crucial piece of our iterative design process. Interventionists and 
classroom teachers tend to perceive meaningful effects of our curriculum (i.e., FUSION as well 
as other mathematics interventions and core curricula we’ve developed) on student outcomes. 
Proximal assessments capture these effects, but standardized measures tend not to reflect the 
same level of impact. 

We are interested in whether - and to what extent — our Pilot Study results reveal differential 
effects for proximal and distal outcomes and the degree to which measured outcomes align with 
teachers’ perceptions of the program’s utility. Such findings will inform the conversation among 
education researchers and the public, more generally, about the nature of research in authentic 
school settings and the relative importance that should be placed on different measures of a 
program’s success. Increased consistency within - and across - studies may come from a greater 
focus on program intensity (i.e., dosage) or a more fine-tuned screening process or assessment 
net. Unfortunately, such emphases may not be deemed feasible in schools hard-pressed to fund 
or staff even existing levels of support. Our results and the resulting conversations will be useful 
to our team as we move forward with other curricula design and implementation efforts and as 
we finalize plans to propose - and conduct - a subsequent FUSION efficacy study. 
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